472 ct pass #2

Kangji · 2021-08-15T08:27:44Z

Major changes:

Added new type of communication channel(Partial Shuffle)
Implemented pass that inserts accumulator vertex, which performs hierarchical aggregation during shuffle.
Implemented unit tests

Minor changes to note:

Tests for the changes:

Other comments:

Network hierarchy information is in bin/network_profiling/labeldict.json
(See PR#315)
Currently, data transfer on partial shuffle communication channel is exactly same with shuffle. It must be implemented.
Need more conditions when applying pass.

Kangji added 8 commits August 15, 2021 15:55

[add] new type of data comm channel

c97de05

[add] new vertex property about network hierarchy

f5b139d

[add] accumulator vertex insertion logic

60537ba

[add] unit test for insertion

2e871b2

[add] implement intermediate combine pass

0a3f0ab

[add] unit tests for ct pass

9fe50a0

[fix] add dependency

c5d6f9c

[fix] fixed labeldict path

32ebe2c

Kangji merged commit 4f137a5 into 472-intermediate-combine Aug 15, 2021

Kangji deleted the 472-ct-pass branch August 15, 2021 08:28

Provide feedback